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EXECUTIVE  SUMMARY 


The  purpose  of  this  executive  summary  is  to  provide  only  an  overview  of  the  Ada  com¬ 
piler  validation  process.  Anyone  who  obtains  services  from  the  Ada  certification  body  must 
understand  the  definition  of  terms  and  follow  the  more  specific  rules  provided  in  the  body 
of  this  document 

•  Organization  and  Responsibilities 

The  Ada  certification  body  consists  of  the  Ada  Joint  Program  Office,  (AJPO),  the  Ada 
Validation  Organization  (AVO),  the  AC  VC  Maintenance  Organization  (AMO),  and 
the  Ada  Validation  Facilities  (AVFs).  The  AJPO,  a  component  of  the  Department  of 
Defense,  establishes  the  policies  of  the  Ada  certification  system,  issues  validation 
certificates  for  AVF-tested  Ada  implementations  and  registers  Ada  implementations 
that  are  un-tested  by  an  AVF.  The  AVO  provides  the  technical  and  administrative 
support  required  to  operate  the  Ada  certification  system.  The  AMO  provides  the  tech¬ 
nical  and  administrative  support  required  to  supply  the  Ada  Compiler  Validation  Ca¬ 
pability  (ACVC)  for  use  in  the  operation  of  the  Ada  certification  system.  There  arc 
five  AVFs  chartered  by  the  AJPO  to  conduct  validation  (see  Appendix  G  for  points 
of  contact).  The  Ada  certification  body  works  with  the  U.S.  Department  of  Com¬ 
merce  which  has  the  responsibility  for  establishing  and  maintaining  a  certification 
system  for  the  Federal  Information  Processing  Standards  (FIPS). 

•  The  Ada  Compiler  Validation  Capability 

The  ACVC  is  designed  to  demonstrate  the  compliance  of  an  Ada  implementation 
with  the  Ada  Programming  Language.  Each  new  version  of  the  ACVC  test  suite  is 
available  for  at  least  six  months  before  it  is  used  for  validation.  The  final  ACVC  ver- 
sionfor  Ada83  is  version  1.11.  Version  1.11  will  be  used  for  validation  until  after  the 
Ada9X  ACVC  is  available.  The  AJPO  will  determine  the  expiration  date  of  ACVC 
1.11  after  Ada  9X  has  been  adopted  by  ANSI.  The  AJPO  will  announce  the  expiration 
date  at  least  six  months  in  advance. 

Some  test  programs  may  contain  test  objectives  which  are  irrelevant  for  a  particular 
Ada  implementation  and  may  be  declared  inapplicable,  in  whole  or  in  parts,  for  that 
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implementation.  The  grading  criteria  for  each  test  and  the  ACVC  User’s  Guide,  dis¬ 
tributed  with  the  ACVC,  provide  information  about  the  applicability  of  test  programs. 
Some  test  programs  are  designed  to  make  use  of  implementation  dependent  charac¬ 
teristics  and  must  be  adjusted  according  to  instructions  given  in  the  ACVC  User’s 
Guide.  Tests  may  be  withdrawn  from  the  ACVC  by  the  AVO  when  it  is  found  that 
they  am  based  on  assumptions  that  need  not  hold  true  for  all  Ada  implementations  or 
that  the  test  program  does  not  meet  its  test  objective.  Any  interested  party  may  dispute 
a  test  program  to  the  Ada  Rapporteur  Group,  ISO  WG9.  An  A  VF  Customer  must  dis¬ 
pute  a  test  program  only  through  an  AVF.  (See  Sections  4.4, 5.2.3, 5.2.4  and  Appen¬ 
dix  B  and  C).  The  AVF  customizes  an  ACVC  for  each  Ada  implementation  which  is 
validated  by  testing.  An  Ada  implementation  passes  a  given  ACVC  version  if  it  pro¬ 
cesses  each  test  of  the  customized  ACVC  and  no  test  is  failed;  otherwise,  the  Ada  im¬ 
plementation  fails  the  ACVC.  The  ACVC  is  available  to  the  public  from  the  National 
Technical  Information  Service  (NTIS)  according  to  Department  of  Commerce  poli¬ 
cies  and  rules  for  the  payment  of  fees  and  for  export  control.  The  ACVC  is  also  avail¬ 
able  to  customers  of  an  AVF. 


♦  Validation  by  Testing 

There  are  well  defined  steps  which  must  be  completed  by  a  customer  and  the  Ada  cer¬ 
tification  body  so  that  the  customer  obtains  a  validation  certificate  and  a  Validation 
Summary  Report  (VSR).  These  steps  are: 


a.  A  formal  validation  agreement  between  the  customer  and  an  AVF  is  required  to 
obtain  validation  services  (see  Section  5.1). 

b.  Prevalidation  which  consists  of  customer  testing,  submission  of  results  to  the 
AVF,  and  the  resolution  of  test  issues  (see  Section  5.2). 

c.  Validation  testing  performed  by  an  AVF  at  a  mutually  agreed  upon  site  (see  Sec¬ 
tion  5.3). 


d.  A  Declaration  of  Conformance  is  completed  and  signed  by  the  customer  not  later 
than  at  validation  testing.  A  validation  certificate  will  not  be  issued  until  a  Dec¬ 
laration  of  Conformance  has  been  completed  (see  Section  5.4  and  Appendix  A). 

e.  A  VSR  is  prepared  by  the  AVF  to  document  the  validation  by  testing  (see  Section 
5.5). 

f.  A  Validation  Certificate  is  issued  by  authority  of  the  AJPO  for  a  successfully  - 

tested  Ada  implementation  (see  Section  5.6). 
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g.  An  Ada  implementation  that  fails  one  to  ten  ACVC  test  programs  may  be  provi¬ 
sionally  validated  for  a  12- month  period.  During  this  year,  the  Ada  implementa¬ 
tion  must  be  re-tested  by  the  AVF  and  successfully  pass  the  ACVC  to  obtain  a 
validation  certificate.  (See  Section  5.7.) 

•  Validation  by  Registration 

An  Ada  implementation  may  be  derived  from  one  which  has  been  validated  by  testing 
(a  base  implementation)  when  four  conditions  are  true  (see  Section  6.2).  These  con¬ 
ditions  provide  an  easy  test  for  determining  whether  to  proceed  with  a  request  to  reg¬ 
ister  an  Ada  implementation  which  was  derived  from  a  base  implementation.  A 
registration  request  must  be  submitted  to  the  AVO  in  the  form  provided  in  Appendix 
D.  The  AVO  will  review  registration  requests  for  completeness  and  plausibility  of  in¬ 
formation.  A  registration  request  which  is  accepted  by  the  AVO  will  be  forwarded  to 
the  AJPO  to  be  added  to  the  public  list  of  Ada  implementations  validated  by  registra¬ 
tion.  A  derived  implementation  loses  its  status  of  being  validated  if  it  is  challenged 
successfully  (see  Section  6.7),  upon  expiration  of  the  validation  certificate  of  its  base 
implementation,  or  when  registration  is  revoked  by  the  customer.  A  procedure  is  pro¬ 
vided  for  adjudication  of  a  claim  that  the  derived  implementation  fails  the  ACVC 
used  to  validate  the  base  implementation  (See  Section  6.7-6.9). 

Three  appendices  have  been  added  to  this  document  Appendix  C  provides  a  descrip¬ 
tion  of  the  Test  Dispute  and  Resolution  Process.  Appendix  D  provides  a  standard 
form  that  will  be  used  for  submitting  a  registration  request  Appendix  F  provides 
Project  Guidelines  for  the  use  of  baselined  Ada  compilers.  These  guidelines  are  nor¬ 
mative  for  the  DoD  (United  States)  and  optional  for  others. 
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1.  INTRODUCTION 

The  United  States  Department  of  Defense  (DoD)  sponsored  the  development  of  the 
Ada  programming  language  and  established  the  Ada  Joint  Program  Office  (AJPO)  as 
part  of  an  effort  to  support  recognized  principles  of  software  engineering  for  a  wide  range 
of  applications.  In  view  of  the  well  known  benefits  of  standardization,  the  AJPO  has  estab¬ 
lished  a  certification  system  to  prevent  the  proliferation  of  dialects  of  the  Ada  program¬ 
ming  language  and  to  encourage  Ada  implementations  which  conform  to  the  [ANSI  83]. 
The  Ada  certification  system  rules  of  procedure  and  management  address  the  validation 
of  Ada  implementations  by  testing  and  by  registration.  This  document  provides  an  oper¬ 
ational  definition  of  a  validated  Ada  compiler  which  is  required  by  [DoD  91]  and  by  the 
[FIRMR  87].  This  version  [3.1]  pertains  to  the  current  validation  process  of  Ada  imple¬ 
mentation  as  defined  by  [ANSI  83],  as  well  as  its  successor,  informally  known  as  Ada  9X. 
Subsequent  revisions  will  reflect  the  Ada  9X  transition  policy  and  procedures  for  use  of  the 
Ada9X  ACVC. 

The  principals  of  the  certification  body  of  the  Ada  certification  system  consist  of  the 
AJPO  for  overall  direction,  the  Ada  Validation  Organization  (AVO)  and  ACVC  Main¬ 
tenance  Organization  (AMO)  for  technical  support,  and  the  Ada  Validation  Facilities 
(AVFs)  for  performing  validations.  The  Ada  certification  body  operates  in  conjunction 
with  the  U.S.  Department  of  Commerce  which  has  the  responsibility  for  establishing  and 
maintaining  a  certification  system  for  the  Federal  Information  Programming  Standards 
(FIPS). 

It  is  important  to  note  the  scope  and  intent  of  Ada  validation.  Users  of  an  Ada  imple¬ 
mentation  are  cautioned  that  the  purpose  of  validation  is  to  encourage  conformity  of  Ada 
implementations  with  the  standard  and  that  characteristics  other  than  those  specified  by 
the  standard,  such  as  performance  or  suitability  for  a  particular  application,  are  outside  the 
scope  of  Ada  validation.  Moreover,  users  are  cautioned  that  the  yardstick  of  conformity 
testing  is  the  collection  of  test  programs  contained  in  the  ACVC.  Thus,  conformity  is  mea¬ 
sured  only  within  the  limitation  of  these  tests. 

A  glossary  of  terms  used  in  this  document  is  provided  in  Section  2.  Terms  defined  in 
the  glossary  are  signified  in  the  text  of  the  document  by  bold  print  Appendices  to  this  doc¬ 
ument  provide  examples  of  documents  used  in  validation,  a  description  of  the  test  dispute 
and  resolution  process,  project  guidelines  for  use  of  baselined  Ada  compilers,  current 
points  of  contacts,  and  references. 
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2.  GLOSSARY  OF  TERMS 

Ada  PROGRAMMING  LANGUAGE:  The  language  defined  by  reference  [ANSI  83]  or 
its  successors. 

ACVC  MAINTENANCE  ORGANIZATION  (AMO):  The  part  of  the  certification  body 
that  maintains  the  ACVC. 

ACVC  Reviewers:  A  part  of  the  certification  body  which  provides  technical  expertise  for 
ACVC  development 

ACVC  TEAM:  The  group  that  produces  the  ACVC  under  contract  to  the  AMO. 

ACVC  USER’S  GUIDE:  A  document  that  explains  the  technical  details  of  processing  the 
test  programs  and  evaluating  their  results. 

Ada  COMPILER:  The  software  and  any  needed  hardware  that  have  to  be  added  to  a  given 
host  and  target  machine  to  allow  transformation  of  Ada  programs  into  executable 
form  and  execution  thereof. 

Ada  COMPILER  VALIDATION  CAPABILITY  (ACVC):  The  means  for  testing  com¬ 
pliance  of  Ada  implementations,  consisting  of  the  test  suite,  the  support  programs, 
the  ACVC  user’s  guide  and  the  template  for  the  validation  summary  report 

Ada  IMPLEMENTATION:  An  Ada  compiler  with  its  host  machine  and  its  target  ma¬ 
chine. 

Ada  JOINT  PROGRAM  OFFICE  (AJPO):  The  part  of  the  certification  body  that  pro¬ 
vides  policy  and  guidance  for  the  Ada  certification  system. 

Ada  Rapporteur  Group  (ARG):  The  Ada  Rapporteur  Group  (ARG)  is  a  subgroup  of  ISO- 
IEC/JTC1/SC22/WG9,  the  International  Standards  Organization  Working  Group  for 
Ada.  Members  of  the  ARG  are  appointed  by  the  convener  of  the  ISO  working  group 
for  the  purpose  of  resolving  issues  with  respect  to  the  interpretation  of  the  Ada  pro¬ 
gramming  language. 

Ada  VALIDATION  FACILITY  (AVF):  The  part  of  the  certification  body  that  carries 
out  the  procedures  required  to  establish  the  compliance  of  an  Ada  implementation. 

Ada  VALIDATION  ORGANIZATION  (A VO):  The  part  of  the  certification  body  that 
provides  technical  guidance  for  operations  of  the  Ada  certification  system. 
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ADAPTIVE  MAINTENANCE:  [ANSI/IEEE  90]  Maintenance  performed  to  make  a  soft¬ 
ware  product  usable  in  a  changed  environment 

APPLICABLE  ACVC  TEST:  A  test  that  is  neither  inapplicable  nor  withdrawn.  Compare 
with  inapplicable  test  program  and  withdrawn  test  program. 

BASE  IMPLEMENTATION:  An  Ada  implementation  that  was  validated  by  testing 
(see  Section  5). 

BASELINE:  [IEEE  90]  A  specification  or  product  that  has  been  formally  reviewed  and 
agreed  upon,  that  thereafter  serves  as  the  basis  for  further  software  development  or 
maintenance  work,  that  can  be  changed  only  through  formal  change  procedures.  (See 
Appendix  F). 

CERTIFICATION  BODY:  [ISO/IEC  86]  An  impartial  body,  governmental  or  non-gov¬ 
ernmental,  possessing  the  necessary  competence  and  reliability  to  operate  a  certifica¬ 
tion  system,  and  in  which  the  interests  of  all  parties  concerned  with  the  functioning 
of  the  system  are  represented. 

CERTIFICATION  MARK:  A  mark  which  may  be  used  only  on  products  directly  asso¬ 
ciated  with  the  Ada  compiler  for  which  the  certification  mark  was  awarded. 

CERTIFICATION  SYSTEM:  [ISO/IEC  86]  A  system  having  its  own  rules  of  procedure 
and  management  for  carrying  out  conformity  certifications. 

COMPLIANCE  of  an  Ada  IMPLEMENTATION:  The  ability  of  the  implementation  to 
pass  an  ACVC  version.  [Note:  For  the  purposes  of  this  document,  compliance  is  a 
practical  measure  of  conformity.] 

COMPUTER  SYSTEM:  [IEEE  90]  A  system  containing  one  or  more  computers  and  as¬ 
sociated  software. 

CONFIGURATION  MANAGEMENT:  [IEEE  90]  A  discipline  applying  technical  and 
administrative  direction  and  surveillance  to:  identify  and  document  the  functional  and 
physical  characteristics  of  a  configuration  item,  control  changes  to  those  characteris¬ 
tics,  record  and  report  change  processing  and  implementation  status,  and  verify  com¬ 
pliance  with  specific  requirements.  (See  Appendix  F). 

CONFORMITY:  [ISO/IEC  86]  Fulfillment  by  a  product,  process  or  service  of  all  require¬ 
ments  specified.  [Note:  Also  see  Section  1.1.2  in  the  ANSI/MEL-STD-1815A] 

CONFORMITY  TESTING:  The  process  described  in  Section  5  of  this  document 


3 


UNCLASSIFIED 


UNCLASSIFIED 


CORRECTIVE  MAINTENANCE:  [ANSI/IEEE  90]  Maintenance  performed  specifical¬ 
ly  to  overcome  existing  faults. 

CUSTOMER:  An  individual  or  corporate  entity  who  enters  into  an  agreement  with  an 
AVF  that  specifies  the  terms  and  conditions  for  AVF  services  (of  any  kind)  to  be  per¬ 
formed. 

CUSTOMIZED  TEST  SUITE:  The  AC  VC  tests,  adjusted  as  necessary,  that  must  be  used 
for  a  given  Ada  implementation  (see  Section  4  J). 

DECLARATION  of  CONFORMANCE:  A  formal  statement  from  a  customer  assuring 
that  conformity  is  realized  or  attainable  on  the  Ada  implementation  for  which  vali¬ 
dation  status  is  requested,  [see  Appendix  A  for  the  format  of  a  declaration  of  con¬ 
formance.] 

DERIVED  IMPLEMENTATION:  An  Ada  implementation  that  was  obtained  from  a 
base  implementation  that  has  a  current  validation  certificate  (see  Section  6). 

FAIL  AN  ACVC  VERSION:  The  Ada  implementation  fails  one  or  more  test  of  the  cus¬ 
tomized  test  suite. 

FAST  REACTION  TEAM  (FRT):  The  part  of  the  certification  body  that  provides  ex¬ 
pertise  for  the  expeditious  resolution  of  test  issues.  (See  Appendix  C). 

HOST  MACHINE:  [IEEE90]  (1)  A  computer  used  to  develop  software  intended  for  an¬ 
other  computer,  (contrast  with  Target  Machine(l))  (2)  A  computer  used  to  emulate 
another  computer.  (3)  The  computer  on  which  a  program  or  file  is  installed.  (4)  In  a 
computer  network,  a  computer  that  provides  processing  capabilities  to  users  of  the 
network. 

INAPPLICABLE  TEST:  A  test  that  contains  one  or  more  test  objectives  found  to  be  ir¬ 
relevant  for  the  given  Ada  implementation. 

INSTRUCTION  SET:  [IEEE90]  The  complete  set  of  instructions  recognized  by  a  given 
computer  or  provided  by  a  given  programming  language. 

KERNEL:  [IEEE90]  (1)  That  portion  of  an  operating  system  that  is  kept  in  main  memory 
at  all  times.  (2)  A  software  module  that  encapsulates  an  elementary  function  or  func¬ 
tions  of  a  system. 

OPERATING  SYSTEM:  [IEEE90]  A  collection  of  software,  firmware,  and  hardware  el¬ 
ements  that  controls  the  execution  of  computer  programs  and  provides  such  services 
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as  computer  resource  allocation,  job  control,  input/output  control,  and  file  manage¬ 
ment  in  a  computer  system. 

PASS  AN  ACVC  VERSION:  No  test  of  the  customized  test  suite  is  failed. 

PERFECTIVE  MAINTENANCE:  [ANSI/IEEE  90]  Maintenance  performed  to  improve 
performance  or  maintainability. 

PREVALIDATION  TESTING:  Processing  of  an  appropriately  customized  test  suite  by 
the  customer. 

PROJECT  COMPILER  A  validated  Ada  compiler  that  is  baselined  for  a  project  in  ac¬ 
cordance  with  applicable  configuration  management  practices.  A  project  compiler 
may  be  used  for  the  life  of  the  project  (see  Appendix  F). 

PROVISIONAL  VALIDATION  CERTIFICATE  (PVC):  Issued  by  authority  of  the 
AJPO  for  tested  Ada  implementations  that  fails  an  ACVC  version.  (See  Section  5.7.) 

REGISTRATION  REQUEST:  A  formal  request  for  extension  of  validated  status  to  a  de¬ 
rived  implementation.  (See  Appendix  D  for  the  required  form). 

RESULT  PROFILE:  The  result  of  processing  the  customized  test  suite  according  to  giv¬ 
en  evaluation  criteria  (see  Section  6). 

SOFTWARE  MAINTENANCE:  [ANSI/IEEE  83]  Modification  of  a  software  product  af¬ 
ter  delivery  to  correct  faults,  to  improve  performance,  or  to  adapt  the  product  to  a 
changed  environment 

TARGET  MACHINE:  [IEEE90]  (1)  The  computer  on  which  a  program  is  intended  to  run. 
(2)  A  computer  being  emulated  by  another  computer. 

TARGET  RUN-TIME  SYSTEM:  The  set  of  sub-programs  that  may  be  invoked  by  link¬ 
ing,  loading,  and  executing  object  code  generated  by  an  Ada  compiler.  If  these  sub¬ 
programs  use  or  depend  upon  the  services  of  an  operating  system,  then  the  target 
run-time  system  includes  those  portions  of  that  operating  system. 

TEST  ISSUE:  Any  problem  arising  during  validation  (see  Section  5.2.3). 

VALIDATION:  The  process  of  checking  the  conformity  of  an  Ada  compiler  to  the  Ada 
programming  language  and  of  issuing  a  certificate  for  the  implementation. 

VALIDATED  Ada  IMPLEMENTATION:  An  Ada  implementation  that  has  been  vali¬ 
dated  successfully  either  by  AVF  testing  (see  Section  5)  or  by  registration  (see  Section 
6). 
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VALIDATED  Ada  COMPILER:  The  compiler  of  a  validated  Ada  implementation. 

VALIDATION  CERTIFICATE  (VC):  Issued  by  authority  of  the  AJPO  for  tested  Ada 
implementations  that  pass  an  AC  VC  version. 

VALIDATION  SUMMARY  REPORT  (VSR):  A  report  produced  by  an  AVF  containing 
results  that  are  observed  from  testing  a  specific  Ada  implementation  or  grouping  of 
Ada  implementations. 

WITHDRAWN  TEST:  A  test  found  to  be  incorrect  and  not  used  in  conformity  testing. 
A  test  may  be  incorrect  because  it  has  an  invalid  test  objective,  fails  to  meet  its  test 
objective,  or  contains  erroneous  or  illegal  use  of  the  Ada  programming  language. 
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3.  ORGANIZATION  AND  RESPONSIBILITIES 

This  section  specifies  the  role  of  organizations  which  form  the  certification  body,  of 
customers  who  receive  service  from  them,  and  of  project  managers  who  use  Ada  imple¬ 
mentations  to  develop  or  maintain  software. 


3.1  Ada  Joint  Program  Office  (AJPO) 

The  AJPO  establishes  the  policies  of  the  certification  system  by: 

a.  setting  validation  standards  to  be  followed  by  all  AVFs; 

b.  establishing  the  conditions  for  issuance,  the  life,  and  the  scope  of  a  validation 
certificate; 

c.  establishing  the  schedule  for  issuing  versions  of  the  ACVC; 

d.  approving  the  release  of  an  ACVC  version; 

e.  designating  members  of  the  certification  body; 

f.  resolving  issues  that  may  arise  during  validation  when  these  issues  can  not  be 
resolved  through  the  best  efforts  of  the  AVO  and  AVF; 

g.  maintaining  the  official  lists  of  validated  Ada  implementations;  and 

h.  issuing  documents  pertaining  to  validation. 


3.2  The  Ada  Validation  Organization  (AVO) 

The  AVO  provides  the  technical  and  administrative  support  required  to  operate  the  cer¬ 
tification  system  by: 

a.  advising  the  AJPO  and  AVFs  concerning  requirements  for  modification  to  the 
validation  procedures; 

b.  resolving  issues  that  may  arise  during  the  validation  process; 

c.  reviewing  all  Validation  Summary  Reports  (VSRs)  prepared  by  AVFs; 

d.  recommending  to  the  AJPO  issuance  of  a  validation  certificate  for  Ada  imple¬ 
mentations  validated  by  testing  (see  Section  5)  and  the  registration  of  derived 
implementations  (see  Section  6); 
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e.  participating  in  the  ACVC  quality  control  and  configuration  management  pro¬ 
cess; 

f.  deciding  on  the  withdrawal  of  test  programs  from  the  ACVC  version  that  is 
being  used  for  validation;  and 

g.  convening  meetings  of  the  members  of  the  certification  body  at  appropriate 
intervals  to  discuss  the  validation  process  and  to  evaluate  practices. 

3.3  The  ACVC  Maintenance  Organization  (AMO) 

The  AMO  and  its  ACVC  Team  provide  the  technical  and  administrative  support  re¬ 
quired  to  supply  the  ACVC  for  use  in  the  operation  of  the  certification  system  by: 

a.  producing  an  ACVC  version  according  to  a  schedule  established  by  the  AJPO; 

b.  performing  ACVC  quality  control  and  configuration  management; 

c.  distributing  an  ACVC  version  to  AVFs  and  the  AVO; 

d.  distributing  an  ACVC  version  to  the  U.S.  National  Technical  Information  Ser¬ 
vice  (NTIS),  a  service  of  the  U.S.  Department  of  Commerce,  for  further  distri¬ 
bution  to  the  public;  and 

e.  providing  information  to  the  public  concerning  the  test  objectives  and  number  of 
test  programs  in  each  version  of  the  test  suite,  and  other  information  that  pro¬ 
motes  a  public  awareness  of  the  test  suite  and  evaluation  criteria. 

3.4  Ada  Validation  Facilities  (AVFs) 

An  AVF  is  chartered  by  the  AJPO  to  conduct  validation  by: 

a.  adhering  to  validation  procedures  approved  by  the  AJPO; 

b.  producing  the  VSR; 

c.  forwarding  unresolved  test  issues  to  the  AVO  for  review  and  analysis,  with  final 
resolution  to  be  provided  by  the  AJPO,  if  necessary; 

d.  providing  advice  on  a  customer’s  registration  request  for  a  derived  Ada  imple¬ 
mentation;  and 

e.  striving  to  satisfy  national  accreditation  criteria. 
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The  AJPO  may  issue  an  AVF  charter  to  an  organization  that  has  been  recognized  as  an 
accredited  testing  laboratory  by  the  U.S.  Department  of  Commerce,  National  Institute  of 
Standards  and  Technology  (NIST).  The  AJPO  may  issue  a  charter  to  an  organization  lo¬ 
cated  in  a  country  which  has  a  Memorandum  of  Understanding  (MoU)  with  the  U.S.  gov¬ 
ernment  covering  the  chartering  of  AVFs,  according  to  the  rules  specified  in  the  MoU.  An 
AVF  charter  may  remain  in  effect  indefinitely;  however,  a  charter  can  be  revoked  by  the 
AJPO,  at  any  time,  for  due  cause.  The  AJPO  may  direct  an  impartial  body  to  conduct  an 
audit  at  any  time  or  prior  to  issuing  an  AVF  charter.  Audits  are  conducted  in  accordance 
with  procedures  established  by  the  AJPO  at  the  time  of  the  audit  and  are  tailored  to  reflect 
the  purpose  of  the  audit. 


3.5  AC  VC  Reviewers 

The  AC  VC  Reviewers  is  the  group  chartered  by  the  AJPO  to  address  quality  and  con¬ 
figuration  management  issues  by: 

a.  assisting  in  refining  overall  testing  philosophy  and  priorities  for  test  coverage; 

b.  providing  expert  technical  review  of  the  test  objectives  and  tests  during  their 
development; 

c.  ensuring  that  advances  in  the  interpretation  of  the  Ada  Programming  Lan¬ 
guage  are  reflected  appropriately  in  the  test  objectives; 

d.  providing  liaison  between  the  certification  system  and  the  ISO  Working  Group 
for  the  Ada  Standard  (i.e.  ISO-IEC/JTC1/SC22/WG9). 

e.  reviewing  and  acting  upon  comments  received  from  compiler  vendors  and  other 
interested  parties. 

Accordingly,  the  ACVC  Reviewers  cooperate  closely  with  the  AVO  and  the  AMO.  In 
particular,  the  ACVC  Reviewers  monitor  the  resolution  of  test  program  disputes  (see  Sec¬ 
tion  5.2.3). 

3.6  Fast-Reaction  Team  (FRT) 

The  Fast-Reaction  Team  is  a  small  group  of  Ada  language  experts  who  advise  the 
AVO  on  complex  test  issues  by: 
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a.  assisting  the  AVO  to  issue  a  decision  to  the  customer  and  AVF  in  a  timely  man¬ 
ner,  and, 

b.  contributing  to  ISO  Working  Group  9  language  issue  resolution. 


3.7  Customers 

Customers  are  serviced  by  the  Ada  certification  body  in  matters  concerning  Ada  val¬ 
idation.  In  requesting  services  of  the  Ada  certification  body,  customers  are  to  provide  ac¬ 
curate  and  complete  information  to  perform  validation,  to  register  derived  Ada 
implementations,  or  to  obtain  other  services. 


3.8  Project  Manager 

Project  managers  are  responsible  for  the  acquisition  of  an  Ada  compiler  that  is  vali¬ 
dated  according  to  the  procedures  set  forth  in  this  document  The  Ada  compiler  selected 
for  software  development  or  for  maintenance  of  existing  software  should  be  baselined  as 
a  project  configuration  item  which  will  be  controlled  by  the  project  configuration  man¬ 
agement  process.  A  project  manager  should  also  obtain  a  copy  of  the  ACVC  used  to  val¬ 
idate  the  project  compiler  so  as  to  periodically  test  it  if  customizing  changes  have  been 
made.  The  project  manager  may  perform  the  ACVC  testing  or  obtain  these  services  from 
an  AVF.  With  good  management  practices,  a  project  manager  plays  an  important  role  in 
preventing  the  proliferation  of  dialects  of  the  Ada  programming  language. 
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4.  THE  ADA  COMPILER  VALIDATION  CAPABILITY 

The  AC  VC  is  designed  to  demonstrate  the  conformity  of  an  Ada  implementation 
with  the  standard.  The  ACVC  is  distributed  as  a  collection  of  test  programs,  support  pro¬ 
grams  which  facilitate  processing  the  tests,  and  an  ACVC  User’s  Guide  that  explains  the 
criteria  for  evaluating  the  results. 

With  Ada  9X  there  will  be  a  shift  toward  tests  for  expected  usage  of  the  language.  Tests 
that  reflect  fringes  of  the  language  in  terms  of  usage,  or  that  reflect  pathological,  remote 
occurrences  will  be  removed.  This  will  permit  the  elimination  of  some  existhg  test  pro¬ 
grams  so  that  coverage  of  additional  features  can  be  achieved  while  maintaining,  or  even 
reducing,  the  size  of  the  ACVC.  The  objective  is  to  place  strict  limits  on  the  size  of  all  fu¬ 
ture  versions  of  the  ACVC. 


4.1  Versions 

A  new  ACVC  version  is  released  periodically  according  to  a  schedule  which  is  deter¬ 
mined  by  the  AJPO.  Each  new  version  incorporates  changes  to  the  ACVC  as  determined 
necessary  by  the  AJPO.  These  changes  are  made  in  order  to  reflect  a  revision  of  the  stan¬ 
dard,  to  incorporate  ISO  WG9  interpretations,  and  to  address  implementer  or  user  com¬ 
ments.  The  test  objectives  for  each  new  ACVC  version  will  be  available  for  public  review 
and  comment  before  test  programs  have  been  implemented.  Comments  on  test  objectives 
should  be  submitted  to  the  ACVC  Reviewers  who  will  recommend  action  to  the  AJPO  af¬ 
ter  consultation  with  the  ACVC  Team,  AMO,  and  the  AVO. 

The  final  ACVC  for  Ada  83  is  version  1.11.  This  version  will  be  used  for  validation 
testing  until  after  the  Ada  9X  ACVC  is  available.  The  expiration  date  for  ACVC  1.11  will 
be  announced  well  in  advance  by  the  AJPO.  Each  new  ACVC  version  will  be  available  to 
the  public  and  AVF  customers  at  least  six  months  in  advance  of  the  date  when  it  will  be 
used  in  validation. 


42  Applicability  of  ACVC  Test  Programs 

Each  ACVC  test  program  has  one  or  more  test  objectives  which  are  described  in  a  com¬ 
ment  in  the  test  program.  Some  test  objectives  might  address  language  features  that  are  not 
required  to  be  supported  by  every  Ada  implementation  (e.g.,  “check  floating-point  oper- 
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adons  for  digits  18”).  These  test  programs  generally  contain  an  explicit  indication  of  their 
applicability  and  the  expected  behavior  of  Ada  implementations  for  which  they  do  not  ap¬ 
ply.  The  determination  of  applicability  is  made  according  to  the  grading  criteria  in  the 
ACVC  or  as  a  ruling  by  the  AVO.  All  applicable  test  programs  must  be  processed  and 
passed  according  to  the  specified  grading  criteria. 


4.3  Test  Modifications 

The  certification  body  strives  to  apply  the  ACVC  as  uniformly  as  practical  to  all  Ada 
implementations.  In  order  to  apply  common  test  objectives  that  depend  on  implementation 
dependent  characteristics  (e.g.,  line  lengths  and  numeric  types),  many  test  programs  must 
be  adjusted  to  a  given  implementation  following  the  procedures  given  in  the  ACVC  User’s 
Guide.  These  adjustments  consist  of  the  insertion  of  implementation  dependent  values  in 
certain  test  programs  at  places  prescribed  by  the  ACVC. 

In  addition  to  the  anticipated  test  modifications,  other  changes  may  be  required  in  order 
to  remove  conflicts  between  a  test  program  and  implementation  dependent  characteristics 
(e.g.,  the  algorithm  for  recovering  from  syntax  errors).  The  allowable  changes  for  each  Ada 
implementation  are  determined  by  an  AVF  after  consultation  with  the  AVO,  except  in  the 
case  of  error-recovery  problems  which  an  AVF  may  address  without  consultation. 


4.4  Test  Withdrawal 

In  any  ACVC  version,  it  is  possible  that  a  test  program  is  based  on  assumptions  which 
need  not  hold  true  for  all  Ada  implementations  or  that  a  test  program  does  not  meet  its 
objective.  In  these  cases,  the  AVO  may  issue  a  correction  to  the  evaluation  criteria  in  the 
ACVC  User’s  Guide  or  the  test  program  may  be  withdrawn  from  that  version  of  the  test 
suite.  Any  interested  party  may  challenge  a  test  program  by  sending  a  rationale  for  the  chal¬ 
lenge  to  the  Ada  Rapporteur  Group  (see  appendix  G  for  the  address).  The  AVO  will  with¬ 
draw  a  test  program  from  the  current  version  of  the  test  suite  when  the  Ada  Rapporteur 
Group  has  accepted  the  challenged  test  for  extended  resolution.  When  an  AVF  customer 
is  preparing  for  validation,  the  customer  must  challenge  a  test  program  only  through  an 
AVF  by  asking  for  a  review  of  its  evaluation  criteria  or  for  its  withdrawal.  The  form  for 
submitting  a  challenge  is  provided  in  Appendix  B. 
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4.5  Customization 

A  customized  test  suite  is  produced  by  the  AVF  for  each  Ada  implementation  that  is 
a  candidate  for  validation.  This  customization  always  consists  of  removing  withdrawn 
tests  and  in  making  required  modifications  to  test  and  support  programs;  and,  it  may  in¬ 
clude  removal  of  some  inapplicable  tests,  when  detailed  rules  for  them  are  provided  in  the 
ACVC  User’s  Guide. 

4.6  Passing  the  ACVC 

An  Ada  implementation  passes  a  given  ACVC  version  if  it  processes  each  test  of  the 
customized  test  suite  in  accordance  with  grading  criteria,  whether  the  test  is  applicable  or 
inapplicable:  otherwise,  the  Ada  implementation  fails  the  ACVC. 


4.7  Availability 

The  ACVC  is  available  to  the  general  public  from  the  NTIS  according  to  U.S.  Depart¬ 
ment  of  Commerce  policies  and  rules  for  payment  of  fees  and  for  export  controL  The 
ACVC  is  also  available  to  a  customer  of  an  AVF  from  that  AVF.  It  should  be  noted  that 
the  distribution  of  the  ACVC  may  be  subject  to  the  nominal  export  restrictions  as  detailed 
by  laws  of  the  U.S.  and  other  countries. 
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5.  VALIDATION  BY  TESTING 

There  are  six  steps  which  must  be  completed  by  a  customer  and  the  certification  body 
so  that  the  customer  obtains  a  validation  certificate  and  a  VSR.  The  same  ACVC  version 
must  be  used  to  complete  the  steps  described  in  this  section.  The  ACVC  version  used  for 
validation  testing  must  be  the  current  one:  there  is  no  exception  to  this  rule.  The  AVF  must 
be  able  to  begin  validation  testing  the  Ada  implementation  at  the  customers  site  before 
the  current  ACVC  version  expires  or  else  validation  with  that  ACVC  version  will  not  be 
allowed.  Anyone  intending  to  obtain  a  validation  certificate  should  contact  an  AVF  with¬ 
out  delay  for  advice  on  the  handling  of  the  ACVC,  on  interpretation  of  the  test  grading  cri¬ 
teria,  and  on  the  operational  details  of  that  AVF’s  management  practices. 


5.1  Step  One:  Validation  Agreement 

In  order  to  obtain  services  from  the  certification  body,  an  interested  party  must  become 
a  customer  of  an  AVF  by  reaching  a  formal  agreement  This  agreement  should  address  the 
following  topics: 

a.  identification  of  the  Ada  implementation  to  be  tested  and  the  ACVC  version  to 
be  used; 

b.  a  statement  of  work,  including  analysis  of  prevalidation  testing,  validation, 
and  preparation  of  the  VSR; 

c.  the  format  of  data  to  be  exchanged; 

d.  a  schedule  of  events  and  the  site  of  validation; 

e.  financial  arrangements; 

f.  retention  of  records; 

g.  AVF  liability;  and 

h.  confidentiality  of  validation  information. 

The  schedule  for  events,  deliverables,  and  payments  should  take  into  account  the  fact 
that  certain  steps  in  the  validation  process  require  interaction  with  other  members  of  the 
certification  body  (i.e.,  AVO  or  AJPO).  The  AVF  will  put  forth  its  best  effort  to  keep  con¬ 
fidential  a  customer’s  intent  to  obtain  a  validation  certificate  and  the  projected  schedule 
for  validation.  This  confidentiality  will  not  be  allowed  to  interfere  with  the  normal  review 
procedures  of  validation.  If  the  customer  requests  confidentiality  for  reasons  of  national 
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security,  the  customer  will  provide  to  the  AVF  an  official  statement  of  the  security  level 
that  applies  to  the  validation,  and  the  AVF  will  obtain  further  guidance  from  the  AJPO. 


5.2  Step  Two:  Prevalidation 

The  requirements  of  this  step  are  discussed  separately  so  that  the  customer  understands 
the  interaction  that  is  required  with  an  AVF. 


5.2.1  Customer  Testing 

After  entering  into  a  formal  agreement,  the  customer  provides  the  necessary  informa¬ 
tion  for  the  AVF  to  prepare  a  customized  test  suite  or,  the  customer  may  prepare  a  cus¬ 
tomized  test  suite  according  to  instructions  in  the  ACVC  user’s  guide.  The  customer  then 
processes  all  the  tests  in  this  customized  test  suite  using  the  candidate  Ada  implementa¬ 
tion  or  another  Ada  implementation  which  produces  the  same  result  If  the  implementa¬ 
tion  provides  for  options  in  the  way  programs  are  processed,  then  the  same  set  of  options 
must  be  chosen  for  all  test  programs,  with  the  possible  exception  of  an  option  controlling 
the  production  of  information  output.  Any  other  exception  constitutes  a  test  issue  which 
must  be  resolved  with  the  AVF  (see  Section  5.2.3).  Test  issues  should  be  sent  to  the  AVF 
for  analysis  as  soon  as  they  are  known. 


5.2.2  Submission  of  Results 

Upon  completion  of  testing,  the  customer  delivers  the  complete  set  of  results  in  the 
agreed  format  to  the  AVF.  These  results  are  accompanied  by  the  following  information: 

a.  a  list  of  test  programs  which  the  customer  claims  are  inapplicable,  together  with 
an  explanation  for  these  claims; 

b.  a  list  of  test  programs  which  are  disputed  but  not  withdrawn  (see  Section  4.4) 
together  with  explanations  (see  Appendix  B  for  format); 

c.  an  annotated  sample  command  script; 

d.  the  complete  set  of  option  settings  used  for  processing  the  customized  test  suite, 
including  the  default  settings;  and 
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e.  complete  and  current  documentation  for  implementation  dependent  characteris¬ 
tics  as  required  in  the  VSR. 


5.23  Test  Issues 

A  test  issue  may  be  any  of  the  following: 

a.  a  missing  or  incomplete  result  to  a  test; 

b.  a  result  presented  in  an  inadequate  form; 

c.  a  disagreement  between  the  customer  and  the  AVF  as  to  the  interpretation  of  a 
result; 

d.  a  change  in  the  choice  of  options  to  be  used  during  testing; 

e.  a  result  which  makes  the  Ada  implementation  fail  the  ACVC  according  to  the 
current  grading  criteria;  or 

f.  an  implementation  dependent  characteristic  that  may  affect  the  conformity  of 
the  Ada  implementation. 

The  material  submitted  by  the  customer  is  analyzed  by  the  AVF  and  test  issues  re¬ 
solved.  If  the  AVF  and  the  customer  cannot  agree  on  a  way  to  resolve  a  test  issue,  the  issue 
will  be  referred  to  the  AVO  for  a  resolution  (see  Section  5.2.4).  It  may  be  justified  to  leave 
a  test  issue  unresolved  at  prevalidation.  For  example,  it  may  be  impossible  to  check  the 
processing  of  control  characters  by  inspecting  printed  results.  The  AVF  will  note  these  un¬ 
resolved  issues  and  describe  the  results  which  will  be  expected  during  validation  testing. 
It  is  also  possible  that  the  customer  information  provided  for  production  of  the  customized 
test  suite  (see  Section  5J2.1)  was  insufficient  so  that  corrections  to  the  customized  test 
suite  must  be  made  and  additional  processing  will  be  required. 


5.2.4  Test  Issue  Resolution 

A  customer  may  challenge  the  applicability  or  correctness  of  any  particular  ACVC  test 
program.  Such  challenges  should  be  presented  to  the  AVF  in  the  test-dispute  format  (see 
Appendix  B).  The  AVF  will  forward  challenges  to  the  AVO  for  resolution;  the  AVO  will 
strive  to  rule  on  the  challenge  within  two  weeks  of  receiving  it  The  AVO  will  forward  all 
challenges  and  rulings  to  die  ACVC  Reviewers,  AMO  and  the  ACVC  Team.  (See  Appen¬ 
dix  C  for  a  description  of  the  Test  Dispute  and  Resolution  Process.) 
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5.2.5  Incomplete  Prevalidations 

The  AVF  and  the  customer  may  agree  that,  at  the  customer’s  risk,  parts  of  the  custom¬ 
ized  test  suite  need  not  be  processed.  The  customer  must  certify  that  the  results  from  a  pre¬ 
vious  prevalidation  submitted  to  the  AVF  or  validation  results  obtained  by  the  AVF  are 
identical  to  those  that  would  have  been  obtained  by  the  customer.  The  normal  practice  is 
to  submit  complete  prevalidation  results. 


5.2.6  Successful  Prevalidation 

Prevalidation  testing  is  successful  if  the  analysis  of  results  and  the  resolution  of  test 
issues  show  that  the  candidate  Ada  implementation  passes  the  customized  test  suite.  Pre¬ 
validation  is  successful  with  caveats  if  the  results  are  satisfactory  except  that  they  were  in¬ 
complete  or  if  resolution  of  some  test  issues  are  deferred  until  validation  testing  by 
agreement  between  the  AVF  and  the  customer. 


53  Step  Three:  Validation  Testing 

Upon  successful  completion  of  prevalidation,  with  or  without  caveats,  the  AVF  wit¬ 
nesses  testing  of  the  Ada  implementation  at  the  site  and  time  mutually  agreed  by  the  AVF 
and  customer.  The  AVF  prepares  a  customized  test  suite  based  upon  customer  informa¬ 
tion  and  any  information  collected  during  the  resolution  of  test  issues.  The  customized  test 
suite  is  installed  and  processed  under  AVF  supervision.  If  the  AVF  determines  that  the  re¬ 
sults  agree  with  those  obtained  from  prevalidation  and  are  satisfactory  with  respect  to  the 
caveats,  the  testing  has  been  successful:  otherwise,  re-testing  will  be  required,  unless  the 
validation  attempt  is  discontinued.  (See  Section  5.7  for  Provisional  Validation.) 


5.4  Step  Four:  Declaration  of  Conformance 

At  some  time  during  the  validation  but  not  later  than  at  validation  testing,  the  custom¬ 
er  will  complete  and  sign  a  declaration  of  conformance.  The  declaration  states  that  the 
organization  which  is  responsible  for  the  production,  maintenance  or  distribution  of  the 
Ada  compiler  is  offering  a  product  that  is  in  compliance  with  the  Ada  programming  lan¬ 
guage.  The  declaration  becomes  part  of  the  AVF  records  and  is  copied  into  the  VSR.  A  Val- 
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idation  Certificate  will  not  be  issued  unless  a  signed  declaration  of  conformance  has 
been  provided  to  the  AVO. 


5.5  Step  Five:  Validation  Summary  Report 

A  VSR  is  produced  for  each  validation  testing  effort.  A  single  VSR  may  cover  valida¬ 
tion  testing  of  several  Ada  implementations,  provided  that  they  all  have  the  same  result 
profile.  The  VSR  provides  the  following  documentation  pertaining  to  the  validation  effort: 

a.  identification  of  the  customer  responsible  for  validation  of  the  Ada  implemen¬ 
tation; 

b.  identification  of  the  organization  responsible  for  the  production,  maintenance,  or 
distribution  of  the  Ada  compiler  or  Ada  implementation; 

c.  identification  of  the  Ada  implementation  tested; 

d.  options  provided  by  the  Ada  compiler  and  identity  of  the  options  used  for  test¬ 
ing; 

e.  the  inapplicable  test  programs  and  implementation  dependent  characteristics 
exhibited  by  the  test  programs  that  established  inapplicability; 

f.  the  implementation  dependent  characteristics  pertinent  to  the  customized  test 
suite; 

g.  description  of  implementation  dependent  characteristics  as  detailed  by  “Appen¬ 
dix  F’  of  [Ada  83  and  its  successor]; 

h.  withdrawn  test  programs; 

i.  modifications  to  test  programs  with  an  explanation  for  such  modifications;  and 

j.  a  description  of  failed  test  programs,  if  applicable.  (See  Provisional  Validation, 
Section  5.7.) 

5.5.1  VSR  Production 

The  VSR  is  prepared  by  the  AVF  but  includes  material  which  is  produced  by  the  cus¬ 
tomer,  such  as  the  “Appendix  F’  required  by  [ANSI  83  and  its  successor].  A  draft  of  the 
VSR  is  sent  to  the  AVO  for  approval  before  or  after  validation  testing.  The  final  version  of 
the  VSR  is  signed  by  the  AVF,  the  AVO,  and  the  AJPO. 
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5.5.2  VSR  Availability 

The  final  version  of  the  VSR  is  available  to  the  general  public  from  NTIS  and  from  the 
AVF  that  produced  it.  The  AVF  may  require  payment  of  a  fee  for  VSR  reproduction  and 
mailing  costs. 

5.6  Step  Six:  Validation  Certificate 

For  each  successful  validation,  one  certificate  is  issued  by  authority  of  the  AJPO.  An 
example  of  a  certificate  is  provided  in  Appendix  C.  Tne  information  on  the  validation  cer¬ 
tificate  describes  the  tested  Ada  implementation:  the  source  of  this  information  is  the 
signed  declaration  of  conformance  which  the  AVF  provides  to  the  AVO  after  completion 
of  testing.  The  customer  will  ensure  that  the  information  contained  on  the  certificate  does 
not  infringe  on  the  rights  of  third  parties  and  may  be  required  to  provide  a  written  statement 
of  consent  from  any  third  party  involved.  Validation  certificates  will  expire  one  year  after 
the  expiration  date  of  the  AC  VC  version  used  for  the  validation.  An  entry  in  the  list  of  Ada 
implementations  validated  by  testing  will  be  made  for  each  certificate  issued.  This  entry 
will  be  removed  when  the  certificate  expires. 


5.7  Provisional  Validation 

When  prevalidation  testing  has  been  unsuccessful,  a  customer  may  petition  the  AVO 
to  continue  with  validation  by  identifying  up  to  ten  AC  VC  test  program  failures  to  dispute 
on  non- technical  grounds.  The  AVO  will  present  the  dispute  to  the  FRT  as  a  petition  for 
validation  with  test  failures.  Unless  the  dispute  is  denied  by  the  FRT,  the  AVO  will  rule 
that  the  test  program(s)  will  be  processed  during  validation  testing  and  the  compiler  be¬ 
havior  will  be  documented  in  the  VSR.  A  Provisional  Validation  Certificate  (PVC)  that 
lists  the  failed  tests  will  be  issued  by  the  AJPO.  A  PVC  will  expire  12  months  after  the  date 
validation  testing  was  completed.  The  customer  will  arrange  with  the  AVF  to  witness  the 
result  of  error  correction  in  the  base  implementation.  The  customer  may  appeal  to  the  ex¬ 
tended  resolution  process  (see  Section  4.4)  but  the  appeal  must  be  accepted  and  the  test  pro¬ 
gram  withdrawn  from  the  ACVC  before  AVF  re-testing.  If  the  compiler  passes  the 
previously  failed  tests,  a  VC  will  be  issued  and  the  VSR  will  be  revised.  The  list  of  PVCs 
will  be  maintained  by  the  AJPO  for  the  public.  VSRs  for  validation  with  failed  tests  will 
be  reviewed  by  the  AVO  and  will  be  publicly  available  from  the  AVF,  AVO  and  AJPO. 
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The  AVO  may  deny  repeated  petitions  from  the  same  customer  for  provisional  validation 
of  an  Ada  implementation. 


5.8  Advertising  Validated  Status 

The  customer  will  not  advertise  or  make  public  claims  that  the  Ada  implementation 
is  validated  until  after  receiving  a  validation  certificate  or  after  receiving  formal  notifica¬ 
tion  from  the  AVF  that  the  AJPO  has  issued  a  validation  certificate.  A  waiver  of  confi¬ 
dentiality  must  be  signed  by  a  customer  who  intends  to  advertise  the  completion  of  events 
that  indicate  progress  toward  completion  of  validation.  If  a  waiver  of  confidentiality  has 
been  signed  with  the  AVF,  the  AVF  will  respond  to  inquiries  about  the  customer’s  adver¬ 
tisements  or  public  claims  by  acknowledging  receipt  of  validation  materials  (i.e.,  a  formal 
agreement,  pre-validation  results,  or  validation  testing  results)  without  judgement  con¬ 
cerning  the  success  of  the  validation. 


5.9  Certification  Mark 

The  certification  mark  (see  Appendix  E  for  reproduction)  may  only  be  used  on  prod¬ 
ucts  directly  associated  with  validated  Ada  compilers,  such  as  disks,  tapes,  packaging,  ad¬ 
vertising,  reference  manuals  and  any  other  associated  documentation  where  a  significant 
portion  relates  to  a  validated  Ada  compiler.  This  unique  mark  distinguishes  compilers  val¬ 
idated  in  accordance  with  the  rules  in  this  document.  The  certification  mark  can  be  repro¬ 
duced  in  any  size,  color,  or  combination  of  colors. 

5.10  Ada  9X  Transition  Period 

Procedures  in  this  document  will  be  used  during  the  transition  to  Ada  9X.  Minor  mod¬ 
ifications  in  procedures  may  be  required  to  reflect  differences  between  the  Ada  83  AC  VC 
and  the  Ada  9X  ACVC.  The  life  of  ACVC  1.11  validation  certificates  and  registrations  has 
been  extended  to  be  compatible  with  ANSI  adoption  of  Ada  9X.  The  Ada  9X  Transition 
Plan  [Ada  9X]  and  Ada  9X  public  reports  provide  current  transition  information. 
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6.  VALIDATION  BY  REGISTRATION 

6.1  Result  Profile 

Two  Ada  implementations  which  pass  a  given  ACVC  version  have  die  same  result 
profile  when: 

a.  they  use  the  same  customized  test  suite; 

b.  inapplicable  test  programs  in  the  customized  test  suite  are  the  same  for  both 
implementations; 

c.  inapplicable  test  programs  are  inapplicable  for  the  same  reasons;  and 

d.  any  implementation  dependent  characteristics  tested  for  by  the  customized  test 
suite  are  the  same  for  both  implementations. 


6.2  Derived  Implementations 

An  Ada  implementation  may  obtain  validated  status  by  registration  when  all  of  the 
following  conditions  are  true: 

a.  The  Ada  compiler  was  obtained  from  the  Ada  compiler  of  the  base  implemen¬ 
tation  by  changes  that  are  within  the  scope  of  accepted  software  maintenance 
practices. 

b.  The  target  machine  of  the  base  and  derived  Ada  implementation  have  compat¬ 
ible  instruction  sets  and  operating  system  or  kernel. 

c.  The  Ada  compiler  has  been  tested  with  the  customized  test  suite  that  was  used 
to  validate  the  base  implementation. 

d.  The  result  profile  for  the  Ada  implementation  is  either  the  same  as  the  base 
implementation  or,  if  there  are  differences,  these  differences  are  justified  as 
being  within  the  scope  of  accepted  software  maintenance  practices,  (see  Section 
6.3). 

Common  examples  of  compatible  instruction  sets  and  operating  systems  are  two  dif¬ 
ferent  computer  system  models  in  a  manufacturer’s  product  line  or  the  computer  systems 
produced  by  different  manufacturers  that  use  the  same  instruction  set  and  operating  sys- 
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terns,  or  any  computer  system  and  a  simulated  or  emulated  computer  system  that  are  the 
same  instruction  set. 

The  changes  that  may  be  made  to  an  Ada  compiler  for  the  purpose  of  derivation  will 
be  within  the  scope  of  software  maintenance  as  applied  to  the  domain  of  compiler  con¬ 
struction.  Changes  must  be  classified  as  corrective,  adaptive,  or  perfective.  Examples  of 
these  changes  are  the  correction  of  a  compiler  error,  the  adaption  to  an  operating  system 
upgrade,  the  transfer  of  the  compiler  to  another  host  machine,  the  addition  of  a  floating 
point  processor  to  a  small  target  machine,  or  the  perfection  of  a  garbage  collection  algo¬ 
rithm.  For  the  purposes  of  obtaining  validated  status  by  registration,  the  changes  required 
to  render  the  base  Ada  compiler  fully  functional  on  a  different  host  machine  or  host  op¬ 
erating  system  is  considered  adaptive  maintenance. 


6.3  Registration  Request 

Any  interested  party  may  initiate  a  registration  by  sending  a  request  to  the  AVO  or  to 
an  AVF.  An  AVF  may  establish  a  fee  for  advisory  service,  or  may  refuse  to  provide  advice 
if  it  did  not  validate  the  base  implementation.  The  registration  request  must  be  signed 
and  provide  the  following: 

a.  reference  to  the  validation  certificate  for  the  base  implementation; 

b.  identifying  description  of  the  Ada  implementation(s)  being  derived  from  the 
base  implementation; 

c.  a  declaration  of  conformance  for  the  derived  implementation(s); 

d.  a  consent  agreement  from  any  other  party  having  a  legal  interest  in  the  Ada  com¬ 
piler; 

e.  a  listing  of  the  Ada  implementation(s)  that  were  tested  with  tin  customized 
test  suite  used  in  the  validation  of  the  base  implementation;  and 

f.  appropriate  evidence  that  the  Ada  implementation^)  may,  in  fact,  be  derived 
from  the  referenced  base  implementation. 

Note  that  the  identifying  description  required  by  b.  must  include  the  nomenclature  of 
the  computer  system(s),  including  operating  system  for  both  host  and  target  machines, 
or  kernel  for  the  target  machine,  if  applicable,  the  Ada  compiler  name  and  version  iden¬ 
tifier,  and  identifier  for  any  components  of  the  host  and  target  machines  listed  in  the  VSR 
for  the  base  implementation.  The  statement,  required  by  f.,  will  include  the  classification 
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of  software  maintenance  changes  (see  Section  6.2)  and  the  effect  these  changes  have  on 
the  result  profile;  will  list  the  differences  between  the  computer  systems  of  the  base  im¬ 
plementation  and  the  derived  implementation;  and  will  describe  the  effect  these  differ¬ 
ences  have  on  the  Ada  compiler.  The  information  required  by  e.  and  f.  substantiate  the 
claim  made  in  the  declaration  of  conformance.  (See  Appendix  D  for  the  proper  form  for 
a  Registration  Request) 


6.4  Evaluation 

The  A  VF  will  not  perform  testing  on  derived  implementations.  Any  AVF  may  review 
the  registration  request  for  completeness  and  the  plausibility  of  information  and  provide 
advice.  The  AVO  will  evaluate  all  registration  requests  before  recommending  that  the  de¬ 
rived  implementation(s)  be  added  to  the  list  of  validated  Ada  implementations 
maintained  by  the  AJPO. 


6.5  Registration 

Registration  requests  which  are  acceptable  to  the  AVO  will  be  registered  as  validated 
Ada  implementations  untested  by  an  AVF.  A  validation  certificate  will  not  be  issued  for 
these  derived  implementations  but  the  customer  may  use  the  certification  mark  award¬ 
ed  to  the  base  implementation.  The  list  of  derived  implementations  and  information  pro¬ 
vided  in  the  registration  request  will  be  available  to  the  public. 


6.6  Expiration  of  Validated  Status 

A  derived  implementation  loses  its  status  of  being  validated  if  it  is  challenged  suc¬ 
cessfully  (see  Section  6.7),  upon  expiration  of  the  validation  certificate  of  its  base  imple¬ 
mentation,  or  when  registration  is  revoked  by  the  customer. 


6.7  Challenges 

Any  derived  implementation  may  be  challenged  by  any  interested  party  through  an 
AVF.  The  challenger  will  pay  a  challenge  fee  to  the  AVF  and  will  submit  a  challenge  re¬ 
quest  which: 
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a.  identifies  the  derived  implementation  being  challenged; 

b.  names  one  ACVC  test  from  the  customized  test  suite  together  with  its  imple¬ 
mentation  dependent  parameters,  if  any;  and 

c.  describes  in  which  way  the  implementation  will  fail  this  test. 

The  AVF  will  send  this  challenge  to  the  originator  of  the  registration  asking  for  com¬ 
ment.  The  challenge  will  be  considered  settled  if  the  registration  is  revoked  by  the  registra¬ 
tion  originator,  otherwise,  the  challenge  will  be  settled  as  outlined  in  Section  6.9. 


6.8  Challenge  Mark 

The  AVF  will  inform  the  AVO  that  a  challenge  for  a  given  derived  implementation 
has  been  received.  The  derived  implementation  will  then  be  marked  as  “challenged”  on 
the  list  of  derived  implementations  maintained  by  the  AJPO.  Information  pertaining  to 
the  challenge  may  be  requested  by  any  interested  party  and  received  from  the  AVF.  It 
should  be  noted  that  a  challenge  mark  applies  only  to  the  derived  implementation  which 
was  named  and  does  not  indicate  any  judgement  about  the  conformity  of  the  challenged 
implementation. 


6.9  Challenge  Test 

The  AVF  will  conclude  a  formal  agreement  with  the  challenger  which  covers  the  AVF’s 
cost  for  performing  a  challenge  test.  For  challenge  testing,  the  challenged  derived  imple¬ 
mentation  will  be  tested  against  the  named  ACVC  test  The  challenger  will  provide  access 
to  the  challenged  derived  implementation  and  appropriate  expertise  to  facilitate  the  AVF 
test.  The  AVO  will  be  informed  of  the  test  result.  Depending  on  its  result,  the  AVO  will  set¬ 
tle  the  challenge  by  either  removing  the  challenge  mark  or  the  derived  implementation 
from  the  list  of  validated  Ada  compilers. 
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Appendix  A. 

Declaration  of  Conformance 


Customer: 

Certificate  Awardee: 
Ada  Validation  Facility: 
ACVC  Version: 


Ada  Implementation 

Ada  Compiler  Name  and  Version:  _ 

Host  Computer  System:  _ 

Target  Computer  System:  _ 

Declaration: 

I  the  undersigned,  declare  that  I  have  no  knowledge  of  deliberate  deviations  from  the 
Ada  Language  Standard  ANSI/MIL-STD-1815A,  ISO  8652-1987,  FIPS  119  as  tested  in 
this  validation  and  documented  in  the  Validation  Summary  Report 


Customer  Signature  Date 

Company 

Tide 


Certificate  Awardee  Signature  Date 

Company 

Tide 


Note:  If  the  Customer  and  the  Certificate  Awardee  are  the  same,  only  the  customer  sig¬ 
nature  is  needed. 
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APPENDIX  B 

Implementer  Dispute  Format 

[part  A] 


Implementer: 

Configuration: 

A  CVC  Version: 

Pre-Validation  Submittal  Date: 


<unplementer's  name> 

<host  &  target  hardware  &  operating  systems> 
<ACVC  version#> 

<due  date  for  in-house  results> 


[Part  A  will  be  completed  once  by  each  implementer,  part  B  will  be  completed  for 
each  dispute.  It  is  not  necessary  for  a  pre-validation  date  to  have  been  established. 
Part  A  information  is  treated  as  confidential.] 
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Reference: 

Summary: 

Discussion: 


[part  B] 

<test  name  {,test  name}> 

<brief  description  of  the  dispute> 
<detailed  description  of  the  dispute> 


[In  this  Discussion,  arguments  should  be  specified  using  test  line  #s  and  references 
to  pertinent  sections  of  the  Ada  standard.  Commentaries  (AI-xxxx),  or  the  ACVC 
Implemented  Guide  (AIG)*.  The  implementer  must  describe  the  behavior  of  the 
implementation  for  the  test  or  tests  that  are  disputed,  stating  the  particular  test  mes¬ 
sages  that  are  produced.  It  is  sufficient  for  the  detailed  description  to  be  limited  to 
the  particular  segment  of  test  code  that  is  disputed.  Relevant  source  code  with  com¬ 
piler  messages  should  be  included.  (For  a  group  of  tests  that  cause  much  the  same 
behavior,  it  is  sufficient  for  a  detailed  description  to  be  given  for  one  of  them,  with 
the  relevant  line  numbers  given  for  the  like  problems  in  the  related  tests.) 

If  the  argument  depends  upon  implementation  constraints  of  hardware  or  software 
(e.g.,  characteristics  of  the  operating  system),  then  these  should  be  specified;  die 
particular  computer  and  operating  system  should  be  identified.  It  is  especially  im¬ 
portant  that  implementations  that  fail  to  pass  some  test  due  to  capacity  limitations 
be  described  in  enough  detail  for  the  A  VO  to  assess  the  reasonableness  of  these  lim¬ 
itations. 

Failure  to  fully  specify  the  points  pertinent  to  a  dispute  might  result  in  an  adverse 
decision  being  made,  with  the  disputer  having  to  further  argue  the  case  with  a  sec¬ 
ond  submittal  to  the  AVO.  Yet  it  is  possible  that  the  Summary  will  suffice  to  ade¬ 
quately  present  a  dispute. 


*.  (The  AIG  is  not  an  official  interpretation  of  the  Ada  standard  but  it  might  provide  useful  information 
in  support  of  a  dispute  in  explaining  implementation  choices.) 
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TEST  DISPUTE  AND  RESOLUTION  PROCESS 

C-l.  Introduction 

A  “dispute”  is  defined  by  the  ACVC  User's  Guide  as  any  result  from  processing  an 
ACVC  test  program  that  is  not  a  passed  or  inapplicable  result  according  to  the  established 
grading  criteria.  This  intentionally  broad  definition  of  a  “dispute”  is  to  make  certain  that 
compiler  implemented  bring  all  deviant  test  results  to  the  attention  of  the  AVF,  without  as¬ 
suming  that  such  results  would  be  accepted  without  special  review.  The  compiler  imple- 
menter  also  provides  a  rationale  for  each  challenge  being  made  to  a  particular  test  program. 
Disputes  are  forwarded  to  the  A  VO,  usually  electronically,  by  the  AVF  on  behalf  of  their 
validation  customer.  For  each  dispute  that  is  accepted  (Le.,  when  the  AVO  rules  in  favor  of 
the  dispute),  it  is  likely  that  some  correction  is  indicated  for  the  disputed  tests.  The  AVO 
withdraws  any  test  that  is  found  to  be  incorrect  to  a  degree  that  makes  it  unsuitable  for  val¬ 
idation.  The  withdrawal  of  a  test  consists  of  including  it  on  a  list  of  tests  that  are  ignored 
for  validations  conducted  with  the  current  ACVC  version.  The  AVO  updates  the  list  of 
withdrawn  tests,  and  distributes  this  list  to  the  AVFs,  the  AMO,  ACVC  Reviewers,  and 
ACVC  Team.  The  AVO  also  maintains  a  database  consisting  of  all  test  disputes  and  their 
resolution  (AVO  rulings)  which  is  periodically  provided  to  the  ACVC  Reviewers,  AMO, 
ACVC  Team,  and  the  FRT.  The  AVO  may  also  recommend  that  certain  resolved  disputes 
be  considered  further  by  the  ARG/URG,  even  though  the  Chairman  of  both  the  ARG  and 
URG  currently  participate  in  deliberations  of  the  FRT. 


C-2.  Expedited  Resolution  Process 

The  AVO  resolves  disputes  by  any  of  three  methods:  a  resolution  that  was  made  previ¬ 
ously  is  applied  to  the  current  dispute  (e.g.,  the  same  dispute  might  be  submitted  at  different 
times  by  different  petitioners);  the  resolution  can  be  determined  unequivocally  based  on  the 
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Ada  standard  or  Ada  Commentaries;  or,  the  resolution  is  determined  based  on  the  deliber¬ 
ations  of  the  FRT.  Although  the  Ada  Compiler  Validation  Procedures  do  not  set  a  limit  on 
the  length  of  time  for  reaching  a  resolution,  the  AVO  attempts  to  resolve  disputes  within 
two  weeks,  an  informal  guideline  that  was  established  by  the  certification  body.  The  AVO 
also  attempts  to  place  priority  on  resolving  disputes  for  AVF  customers  who  have  a  firmly 
scheduled  date  for  validation  testing.  Implementers  should  submit  disputes  well  in  advance 
of  a  scheduled  validation  testing  date,  (see  Section  5) 

On  receipt  of  a  dispute,  the  AVO  checks  whether  the  issue  matches  any  that  had  been 
previously  resolved.  If  the  dispute  is  new,  it  is  given  an  initial  AVO  analysis  which  involves 
research  using  the  Ada  Commentaries  in  conjunction  with  the  Ada  standard  and  references 
to  previous  dispute  deliberations.  A  dispute  is  referred  to  the  FRT  when  questions  of  inter¬ 
pretation  arise  and  a  resolution  is  not  obvious.  The  AVO  presents  the  dispute  and  any  ad¬ 
ditional  information  resulting  from  an  initial  analysis  to  the  HIT  by  e-mail  without 
disclosing  the  identity  of  the  petitioner.  Deliberation  of  the  dispute  proceeds  with  the  ex¬ 
change  of  each  expert's  opinion  and  analysis.  The  AVO  participates  in  the  deliberation  by 
providing  information  as  requested  (e.g.,  ACVC  tests  or  information  from  the  petitioner), 
eliciting  discussion  from  the  experts,  and  making  or  challenging  technical  points  raised  in 
the  discussion.  In  general,  where  an  issue  receives  support  from  some  members  of  the  FRT, 
the  dispute  is  accepted. 

There  is  no  prescribed  formality  to  the  FRT  deliberations,  such  as  voting  procedures  or 
time  limits  on  deliberation.  The  AVO  might  extend  deliberation  when  a  basis  for  resolving 
the  dispute  has  not  been  made.  However,  the  AVO  will  give  its  ruling  on  the  dispute  when 
a  sufficient  basis  has  been  established,  regardless  of  whether  the  FRT  discussion  continues. 


C-3.  Types  of  Resolutions 

The  resolution  of  a  dispute  is  either  an  acceptance  or  rejection  of  the  petitioner's  argu¬ 
ments.  Acceptance  can  result  in  either  withdrawal  of  the  test  program  from  the  ACVC  or 
in  a  “Test,  Processing,  or  Evaluation”  modification  for  validation.  A  dispute  may  be  reject¬ 
ed  if  it  conflicts  with  the  Ada  standard  or  Ada  Commentaries  or  if  it  is  a  test  to  which  pre¬ 
viously  validated  implementations  conform  and  the  petitioner  has  not  provided  compelling 
reasons  for  a  deviation.  A  dispute  may  lead  to  the  withdrawal  of  a  test  program  if  the  test 
is  shown  to  be  incorrect  to  a  degree  that  wrongly  influences  implementation.  Withdrawn 
tests  have  no  effect  on  validation  (they  are  generally  not  processed).  If  the  dispute  shows 
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the  affected  test  program(s)  to  be  incorrect  in  only  a  minor,  limited  degree,  generally  the 
AVO  will  direct  that  the  test(s)  be  processed  with  a  test  modification. 

There  are  three  types  of  test  modification:  Test,  Processing,  and  Evaluation  modifica¬ 
tions.  A  Test  Modification  is  an  actual  change  to  the  code  of  the  test  (e.g.,  adding  a  choice 
to  an  exception  handler).  A  Processing  Modification  is  a  change  to  the  way  in  which  the 
test  is  processed  (e.g.,  re-ordering  the  compilation  of  component  files  of  a  multi-compila¬ 
tion  test).  And  an  Evaluation  Modification  is  simply  the  grading  of  the  observed  results  by 
other  than  the  established  grading  criteria  (e.g.,  interpreting  particular  intermediate  output 
and  a  final  “failed”  result  as  “passed”,  according  to  an  understanding  of  the  dispute).  All 
test  modifications  are  documented  in  the  VSR. 


C-4.  Example  of  Extended  Resolution 

The  major  issue  to  arise  during  fiscal  year  1991  was  the  vulnerability  of  many  tests  to 
optimization  that  removes  assignments  to  unused  variables.  Many  of  the  tests  that  check 
that  an  exception  is  raised  under  prescribed  conditions  use  code  that  does  not  prohibit  the 
exception-raising  expression  from  being  eliminated.  The  Ada  standard  (1 1.6:7)  permits  an 
operation  to  be  eliminated  “if  its  only  possible  effect  is  to  propagate  a[n]  exception”.  In 
many  of  the  tests  this  is  precisely  the  case,  as  the  expression  returns  a  value  that  is  intended 
to  be  assigned  to  a  variable  that  is  never  used  (hence,  the  value  is  not  needed  and  the  as¬ 
signment  and  expression  evaluation  need  not  be  made).  The  AVO  rejected  only  one  case 
of  optimization,  where  it  removed  a  programming  safeguard  (local  variable  initialization 
for  a  block)  and  was  too  far  from  what  the  standard  clearly  permits.  This  issue  has  been 
referred  to  the  ACVC  Team  and  the  ACVC  Reviewers  for  consideration  during  the  devel¬ 
opment  of  the  Ada  9X  ACVC  and  the  Ada  Rapporteur  Group. 

C-5.  Summary 

There  is  no  limit  on  the  number  of  disputed  tests  that  can  be  submitted  by  an  implement- 
er.  Although  there  is  a  risk  that  a  dispute  will  not  be  decided  in  the  implementer’s  favor,  that 
risk  can  be  managed  so  as  to  not  affect  validation  by  early  submission  of  disputes.  The  va¬ 
lidity  of  grading  criteria  for  ACVC  tests  has  been  very  important  for  Ada  83,  and  will  take 
on  even  greater  importance  for  Ada  9X  usage  based  tests.  Any  interested  party  may  dispute 
an  ACVC  test  and  its  grading  criteria.  Disputes  that  lead  to  the  removal  of  unusual  test  cas- 
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es  (informally  called  pathological  tests)  from  the  last  version  of  the  Ada  83  ACVC  will  be 
helpful  in  making  a  transition  to  the  Ada  9X  usage  based  test  coverage.  Future  modifica¬ 
tions  to  procedural  rules  for  test  disputes  will  be  consistent  with  the  Ada  9X  ACVC. 
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APPENDIX  D 

REGISTRATION  REQUEST 


A  registration  request  must  have  two  main  parts:  (1)  a  letter  and  (2)  an  enclosure  con¬ 
sisting  of  specific  information  to  support  the  request 

PART  1  -  LETTER 

The  request  letter  should  be  addressed  to  the  Ada  Validation  Organization  (AVO) 
(note:  see  Appendix  G  for  address)  even  though  the  requester  does  seek  advice  from  an 
AVF.  The  mandatory  parts  of  the  letter  are: 

1.  Certificate  number  and  description  of  the  Base  Implementation, 

2.  List  of  derived  implementations  that  were  fully  tested  with  the  customized 
ACVC  used  in  the  base  validation.  If  this  list  is  less  than  the  list  for  which  reg¬ 
istration  is  requested,  item  4  must  also  be  included  in  this  letter. 

3.  We  declare  that  we  have  no  knowledge  of  deliberate  deviations  from  the  Ada 
Language  Standard  [ ANSI/MIL_STD- 1 8 15 A/ISO  8652- 1987/FIPS  11 9]  in  the 
implementation. 

4.  For  the  derived  implementations  that  were  not  fully  tested,  we  declare  that,  to  the 
best  of  our  knowledge,  these  implementations  will  pass  the  customized  ACVC 
used  in  the  base  validation.  Any  possibly  observed  deviations  are  the  conse¬ 
quence  of  implementation  errors. 

5.  If  this  statement  applies,  legal  interests  of  (name)  are  affected  by  this  registration 
request  and  their  consent  is  documented  in  enclosure  (number)  to  this  letter. 

6.  The  Ada  implementations  for  which  registration  is  requested  may,  in  fact,  be 
derived  according  to  the  rules  given  in  the  Ada  Validation  Procedures,  Version 
3.1  as  explained  in  enclosure  (number). 

7.  Signature 

8.  Corporate  Title 
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PART  2  -  TEMPLATE  FOR  SUPPORTING  INFORMATION 

1.  Identification  of  Base  Ada  Implementation: 

a.  Certificate  number: _ 

2.  Registration  requested  for. 

Note:  This  registration  request  pertains  to  all  Ada  implementations  obtained  by  tak¬ 
ing  the  Ada  compiler  and  selecting  any  host  and  any  target  from  the  list  below. 

Compiler:  [name  and  version(s)/release  number/product(s)  range  of  versions/re¬ 
leases] 

Host:  choose  either  (a)  or  (b) 

a.  [make,  specific  members  of  series  and/or  model  numbers]  (e.g.  IBM  PS2 
model  80-110  &  113)  or, 

b.  [all  members  of  a  model  series]  (e.g.  IBMPS2  all  models,  or  HP  9000  series 
300) 

Operating  System:  [name  and  version/release  numbers)] 

Target:  choose  (a),  (b),  or  (c) 

a.  [make,  specific  members  of  series  and/or  model  numbers]  or 

b.  [all  members  of  a  model  series]  or 

c.  [members  of  an  instruction  set  architecture  and  board  implementation^)] 
(e.g.  Motorola  68020-MVME133-1  &  MVME130COF,  Motorola  68030 - 
MVME147,  Motorola  68030-MilSpec285) 

Operating  System:  choose  either  (a)  or  (b)  or  (c) 

a.  [OS  name  and  version/release  number  for  all  target  machines  listed]  or 

b.  [kernel  identifier  for  all  target  machines  listed]  or 

c.  [none  for  all  target  machines  listed] 

3.  Evidence  that  the  Ada  implementations  listed  above  should  be  validated  by  reg¬ 
istration: 

a.  State  the  type  of  software  maintenance  changes  made  to  the  compiler  and 
describe  what  was  done. 

•  corrective  *  maintenance  performed  specifically  to  overcome  an  existing 
fault. 
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•  perfective  -  maintenance  performed  to  improve  performance  or  main¬ 
tainability. 

•  adaptive  -  maintenance  performed  to  make  a  software  product  usable  in 
a  changed  environment 

b.  Which  host/target  combinations  were  fully  tested  with  the  A CVC? 

Testing  with  “customer  applications”  or  other  types  of  tests  is  not  equivalent  to 
running  the  ACVC.  If  a  sub- set  of  ACVC  tests  were  run,  so  state  and  indicate  the 
sub-set. 

c.  State  whether  the  result  profile  is  the  same  or  different  from  the  base.  If  dif¬ 
ferent,  list  all  ACVC  test  results  that  are  different  or  exhibit  different  behav¬ 
ior  and  explain  the  difference. 

d.  What  is  the  authoritative  source  used  to  determine  the  technical  compatibil¬ 
ity  between  the  derived  implementations  and  the  base? 

If  copies  of  technical  manuals  are  supplied  as  the  source  of  this  information,  give 
the  enclosure  number  where  this  information  can  be  found.  Do  not  duplicate  a  pre¬ 
vious  submission  of  identical  technical  material,  just  reference  its  previous  submis¬ 
sion.  If  there  has  been  a  change  in  the  previously  submitted  material,  submit  only 
the  change.  If  there  is  no  authoritative  source  of  information  given,  this  registration 
request  may  be  refused. 
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APPENDIX  E 


Certification  Mark 


THIS  PRODUCT  CONFORMS  TO 
ANSI/MIL-STD-1 81 5A  AS 
DETERMINED  BY  THE  AJPO 
UNDER  ITS  CURRENT  TESTING 


TESTING  PROCEDURES  THIS  PRODUCT  CONFORMS  TO 

ANSI/M  IL-STD-1815A  AS 
DETERMINED  BY  THE  AJPO 
UNDER  ITS  CURRENT  TESTING 
PROCEDURES 
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APPENDIX  F 


PROJECT  GUIDELINES 

F-l.  BACKGROUND 

The  general  validation  procedures  given  in  the  preceding  sections  of  this  document  de¬ 
scribe  the  process  of  establishing  the  conformity  of  an  Ada  implementation  before  it  is 
offered  for  sale  in  the  market  place.  In  the  U.S.,  Federal  Information  Resources  Manage¬ 
ment  Regulations  [FIRMRs]  require  that  a  compiler  must  have  validated  status  when  it  be¬ 
comes  part  of  the  Federal  Government’s  software  inventory.  The  FIRMRs  also  require  that 
application  software  supplied  to  the  Federal  Government  will  be  the  product  of  a  validated 
compiler.  In  other  countries,  there  may  be  established  government  or  commercial  require¬ 
ments  that  are  similar  to  the  FIRMRs  in  intent  Since  validated  status  is  limited  to  the  life 
of  a  particular  validation  certificate,  suppliers  of  compilers  or  software  must  periodically 
complete  the  steps  in  the  validation  process  so  as  to  have  a  current  validation  certificate 
for  Ada  implementations.  However,  there  are  practical  project-level  difficulties  associat¬ 
ed  with  maintaining  an  always  current  validated  status  for  all  compilers  being  used  to  de¬ 
velop  or  maintain  software.  This  section  provides  guidelines  for  project  managers  that  are 
consistent  with  the  goal  of  preventing  the  proliferation  of  dialects  of  the  Ada  program¬ 
ming  language.  These  guidelines  should  be  tailored  by  organizations  that  manage  software 
projects  according  to  their  quality  assurance  policies  and  configuration  management  pro¬ 
cedures.  These  guidelines  are  normative  for  the  U.S.  DoD  and  optional  for  others. 


F-l.l  Acquisition  Requirements 

A  project  manager  will  identify  the  requirement  for  a  validated  Ada  compiler  as  an 
action  within  the  context  of  project  milestones.  This  requirement  will  be  met  when  a  vali¬ 
dated  Ada  implementation  has  been  delivered  for  the  project  If  the  Ada  implementation 
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selected  for  the  project  is  not  validated,  the  project  manager  is  responsible  for  requesting 
that  validated  status  will  be  obtained  in  accordance  with  the  general  rules  of  procedure  for 
validation  by  testing  or  registration  as  early  as  possible  in  the  software  development  pro¬ 
cess. 


F-1.2  Baselined  Project  Compiler 

The  project  manager  should  determine  whether  the  Ada  compiler(s)  used  on  the 
project  will  be  upgraded  or  replaced  by  the  supplier  on  a  validation  schedule  which  is  in¬ 
dependent  of  project  milestones;  or,  whether  the  validation  certificate  will  be  allowed  to 
expire  before  these  milestones  are  reached.  The  project  manager  will  baseline  the  validated 
compiler  and  the  ACVC  at  a  given  version  when  it  is  more  cost-effective  for  the  project  to 
forego  replacements  of  the  compiler  with  later  validated  versions.  If  the  compiler  being 
used  on  the  project  does  not  have  validated  status  (e.g.,  the  version  was  derived  but  not  reg¬ 
istered  by  the  vendor),  it  must  have  validated  status,  by  testing  or  registration,  to  qualify  as 
a  project  compiler.  When  the  validated  compiler  has  been  baselined,  it  then  becomes  a 
project  compiler  for  the  lifetime  of  that  project,  or  until  the  project  manager  establishes  a 
requirement  for  validation  prior  to  reaching  a  particular  project  milestone.  When  a  vali¬ 
dated  Ada  compiler  has  been  baselined  for  a  project,  configuration  management  proce¬ 
dures  must  be  established  to  ensure  complete  documentation  of  any  derivations  from  it  or 
any  deviations  from  ACVC  test  results  obtained  previously  from  the  baselined  compiler. 


F-1.3  Project  Registration 

The  project  manager  should  notify  the  AJPO  when  a  validated  Ada  implementation 
has  been  selected  for  a  particular  project  as  a  baselined  version  that  will  be  used  for  an  ex¬ 
tended  period  or  for  the  life  of  the  project  Notification  may  take  the  form  of  the  registration 
request  generally  used  by  compiler  vendors  with  the  addition  of  information  that  identifies 
the  project.  (See  Appendix  D  for  the  registration  request  form  and  Section  6  for  general 
rules.)  Experience  has  shown  that  formalizing  the  declaration  of  a  project  compiler  is  use¬ 
ful  for  a  project  manager,  as  well  as  for  the  AJPO,  when  disputes  arise  concerning  the  val¬ 
idated  status  of  compilers  that  have  been  used  on  a  project 
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F-1.4  Embedded  Applications 

Some  software  development  projects  include  applications  that  execute  on  a  target  ma¬ 
chine  that  is  embedded  in  a  larger  system.  A  compiler  used  to  generate  object  code  for  the 
embedded  target  machine  is  considered  to  be  a  validated  project  compiler  only  if  all  of 
the  following  conditions  are  satisfied: 

a.  The  project  compiler  was  a  validated  Ada  compiler  or  was  derived  from  a  val¬ 
idated  Ada  compiler. 

b.  All  mandatory  features  of  the  Ada  programming  language  that  can  be  sup¬ 
ported,  or  are  emulated  on  the  embedded  target  machine,  are  supported  by  the 
compiler  for  the  target  That  is,  compilers  for  the  restricted  target  shall  not  be 
arbitrarily  constrained  to  subset  implementations  of  the  Ada  programming  lan¬ 
guage. 

During  software  development  of  embedded  applications,  project  managers  must  ensure 
that  all  run-time  systems  used  to  generate  code  are  managed  as  configuration  items  con¬ 
sisting  of  libraries  and  documentation  for  all  versions  that  will  be  delivered  for  operational 
use.  All  limitations  of  the  restricted  target  should  be  documented  in  Appendix  F  for  the  Ada 
implementation. 


F-1.5  Re-testing 

The  project  manager  should  ensure  that  the  project  compiler  retains  a  known  degree 
of  compliance  with  the  standard  throughout  the  remaining  life  of  the  project  by  periodical¬ 
ly  re- testing  the  project  compiler  using  the  AC  VC  version  used  to  originally  establish  the 
conformity  of  the  base  implementation  or  a  derived  implementation.  The  project  man¬ 
ager  should  determine  whether  this  testing  will  be  done  by  project  personnel  or  by  an  AVF, 
so  as  to  obtain  a  VSR  that  will  document  the  result  of  re-testing.  The  services  of  an  AVF 
are  desirable  when  project  personnel  are  unfamiliar  with  the  testing  procedure  and  interpre¬ 
tation  of  test  results.  (A  VC  will  not  be  issued  for  a  project  compiler  unless  it  passes  a  cur¬ 
rent  ACVC  version  and  validation  is  conducted  as  described  in  Section  5.) 

When  a  software  release  has  been  produced  on  several  project  compilers,  ACVC  re¬ 
testing  requirements  apply  to  each  of  these  compilers.  Records  of  compiler  maintenance 
changes  and  the  ACVC  result  profile  will  be  maintained  for  each  project  compiler.  These 
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records  will  be  current  and  available  for  inspection  by  the  government  Examination  of 
such  records  should  be  a  routine  action  for  audit  teams. 


F-1.6  Ada  9X  TRANSITION 

The  last  [Ada  83]  ACVC  version  that  can  be  baselined  with  a  project  compiler  is 
AC  VC  version  1.11.  Project  managers  who  decide  to  baseline  a  validated  Ada  compiler 
for  [Ada  83]  must  do  so  before  expiration  of  the  validation  certificates  issued  for  ACVC 
1.11  validations.  The  first  Ada  9X  ACVC  version  that  can  be  baselined  with  an  Ada  9X 
project  compiler  is  ACVC  version  2.1. 
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APPENDIX  G 
Points  of  Contact 


Ada  Validation  Facility  Managers 
Dr.  William  Dashiell 

National  Institute  of  Standards  and  Technology 
National  Computer  Systems  Laboratory 
Building  225, 

Room  A266 

Gaithersburg,  MD  20899 
U.S. 

phone:  301-975-2490 

net:  nist-avf@ajpo.seLcmu.edu 

Captain  Russ  Hilmandolar,  USAF 
Standard  Languages  and  Environments  Division 
Engineering  Applications  Directorate 
DCS/Communications-Computer  Systems 
Wright-Patterson  Air  Force  Base 
Ohio  45433-6503 
phone:  513-255-4472 
net:  hilmanrk@adaivc.wpafb.af.mil 

Mr.  Jon  Leigh 

The  National  Computing  Centre,  Ltd. 

Oxford  Road 
Manchester 
England,  M17ED 
phone:  +44-61-228-6333 
net:  uk-avf@ajpo.sei.cmu.edu 

Mr.  Alphonse  Philippe 
AFNOR 

Tour  Europe,  cedex  7 
F-92080  Paris  la  Defence 
France 

phone:  +33-1-42-91-55-55  or  57-96 
net:  afnor@ajpo.sei.cmu.edu 
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Mr.  Michael  Tonndorf 
IABG,  Dept.  ITE 
Einsteins'  ^0 
W-8012  Ottobrunn 
Germany 

phone:  +49-89-6088-2477 
net:  tonndorf@ajpo.sei.cmu.edu 
(EU  net:  tonndorf@  ite.iabg.de) 

ACV C  Maintenance  Organization 

Ms.  Christine  Anderson  (Ada  9X) 

Ada  9X  Project  Office 
PL/VTET 

Kirtland  AFB,  NM  871 17-6008 
phone:  505-846-0817 
fax:  505-846-2290 
net:  andersonc@uservx.plk.af.mil 

ACVC  Team  (Ada  9X) 

Mr.  Mike  Middlemas 
SAIC 

10770  WaterRidge  Circle 
MS  213 

San  Diego,  CA  92121 
phone:  619-552-5326 
net:  middlema@ajpo.sei.cmu.edu 

ACV C  Reviewers  (Ada  9X) 

Dr.  Nelson  Weiderman  (Chair) 

127  Schooner  Dr. 

Wakefield,  RI 02789 
phone:  401-783-6863 
net:  nhw@sei.cmu.edu 

Ada  Rapporteur  Group  (ISOWG9) 

Dr.  John  Goodenough  (Chair) 

net:  jbg@sei.cmu.edu 

net:  comp.lang.ada@AJPO.sei.cmu.edu 

Ada  Joint  Program  Office 

Dr.  John  P.  Solomond 

Director  Ada  Joint  Program  Office 

Deputy  Director,  Defense  Research  and  Engineering  (S&T) 

Pentagon  Room  3E1 14  (Fern  St.) 

Washington,  D.C.  20301-3081 

phone:  703-614-0209 

net:  solomond@ajpo.sei.cmu.edu 
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Ada  Validation  Organization 

Ms.  Audrey  A.  Hook 
Institute  for  Defense  Analyses 
1801  N.  Beauregard  St. 

Alexandria,  Va.  22311 
phone:  703-845-6639 
net:  hook@ida.org 

Fast  Reaction  Team 

Mr.  Dan  Lehman  (test  challenges) 

Institute  for  Defense  Analyses 
1801  N.  Beauregard  St 
Alexandria,  Va.  22311 
phone:  703-845-6633 
net:  avo@ajpo.sei.cmu.edu 

Ada  Information  Clearinghouse  (for  the  AJPO) 

Ms.  Susan  Carlson  (industry  and  government  inquiries) 
Ada  Information  Clearinghouse 
ITT  Research  Institute 
4600  Forbes  Blvd. 

Lanham,  Md.  20706-4312 
Telephone:  703-685-1477 

ACVC  and  VSR  Distribution 

National  Technical  Information  Service 
U.S.  Department  of  Commerce 
5285  Port  Royal  Road 
Springfield,  Va.  22161 
Telephone:  703-487-4650 

DoD  Public  Affairs  (press  inquiries) 

Ms.  Jan  Walker 
OASDPA 

Pentagon  Room  2E765 
Washington,  D.C.  20301-1400 
Telephone:  703-695-0192 
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Validated  Compiler  Lists 

AJPO  -  Official  Ada  lists,  updated  monthly. 

Michele  Kee 

Ada  Information  Clearinghouse  (for  the  AJPO) 

IIT  Research  Institute 
4600  Forbes  Boulevard 
Lanham,  MD  20706-4312 
Telephone:  703-685-1477 

NIST  -  All  FIPS  validated  compilers,  updated  quarterly. 

National  Institute  of  Standards  and  Technology 
National  Computer  and  Telecommunications  Laboratory 
Gaithersburg,  MD  20899 
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APPENDIX  H 


References 


[Ada  9X]  Ada  9X  Project  Reports,  Office  of  the  Under  Secretary  of  Defense  for 

Acquisition,  Washington,  D.C.  Available  from  the  Ada  9X  Project  Of¬ 
fice. 

[ANSI  83]  American  National  Standards  Institute  and  United  States  Department  of 
Defense:  ANSI/MIL-STD-1815A  Reference  Manual  for  The  Ada  Pro¬ 
gramming  Language,  1983  Note:  This  standard  is  identical  with  ISO/ 
8652-1987  and  FIPS  1 19, 1985.  The  ANSI  83  is  available  from  various 
publishers  and  from  the  U.  S.  Government  at  the  following  addresses: 

Commanding  Officer 
Naval  Publications  and  Forms  Center 
Attn:  NPODS 
5801  Tabor  Avenue 
Philadelphia,  Pennsylvania  19120 

Superintendent  of  Documents 
Government  Printing  Office 
Washington,  D.C.  20402 

The  ISO/8652-1987  (French  and  English  versions)  is  available  from: 
AFNOR 

Tour  Europe,  cedex  7 
F-92080  Paris  la  Defence 
France 

[ANSI/IEEE  90]  American  National  Standards  Institute  and  Institute  of  Electrical  and 
Electronics  Engineers,  Inc.,  Standard  610.12-1990; 44 ANSI/IEEE  Stan¬ 
dard  Glossary  of  Software  Engineering  Terminology”. 
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[DoD  91] 
[FERMR  90] 

[ISO  74] 
[ISO/IEC  91] 


Department  of  Defense  Instruction:  Number  5000.2,  January  1,  1991 
Subject:  Defense  Acquisition  Management  Policies  and  Procedures 

General  Services  Administration,  Federal  Information  Resources  Man¬ 
agement  Regulations  (F1RMR)  201.20.303  and  201.39.1002,  28  De¬ 
cember  1990,  Subject:  Federal  Standards;  and.  Associated  Federal  ADP 
and  Telecommunications  Standards  Index  (updated  semi-annually). 
Subject:  Federal  Information  Processing  Standards;  Implementation, 
Validation  and  Conformance. 

International  Standards  Organization:  ISO  2382/1- 1974  Data  Processing 
-  Vocabulary  -  Section  01:  Fundamental  Terms. 

International  Standards  Organization:  ISO/IEC,  Guide  2,  6th  edition 
1991  -  General  Terms  and  Their  Definitions  Concerning  Standardiza¬ 
tion  and  Related  Activities. 
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